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ABSTRACT 


This  report  reviews  nine  studies  concerned  with  identifying  the  knowledge,  skills,  abilities,  and  other 
traits  (KSAOs)  that  are  required  for  success  as  a  military  pilot.  The  report  begins  with  a  review  of 
taxonomies  used  to  identify  the  required  KSAOs  and  notes  issues  associated  with  their  use. 
Definitional  problems  of  three  attributes  commonly  associated  with  success  as  a  pilot —  Mechanical 
Aptitude,  Timesharing,  and  Situational  Awareness — are  discussed.  Next,  the  studies  that  identified  the 
required  KSAOs  are  reviewed.  Limited  conclusions  can  be  drawn  about  the  required  KSAOs  from  the 
studies  reviewed.  Few  investigators  examined  categories  of  knowledge  as  indicators  of  success. 
Similarly,  the  skills  required  for  success  were  rarely  identified  because  investigators  assumed  they 
would  be  trained.  Problems  with  definitions  of  abilities  and  other  methodological  issues  limited  the 
identification  of  required  abilities  and  restricted  cross-study  comparisons.  Nevertheless,  several 
abilities,  particularly  Perceptual  Speed  and  Spatial  Orientation,  were  closely  associated  with  success  as 
a  military  pilot.  Identification  of  other  traits  required  for  success  was  hampered  by  the  lack  of  strong 
theories  of  personality  to  guide  investigators.  The  report  concludes  with  recommendations  for  the 
development  of  a  new  taxonomy  and  for  research  focused  on  several  attributes  that  may  be  critical  for 
success  as  a  military  pilot. 
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KSAOS  FOR  MILITARY  PILOT  SELECTION:  A  REVIEW  OF  THE  LITERATURE 


This  report  is  part  of  a  contract  entitled  “Historical  Scientific  Analysis  of  Aviation  Selection.” 
This  contract  had  three  goals:  1)  describe  the  development  of  pilot  selection  systems  from  their 
inception  to  the  present,  2)  develop  a  searchable  database  of  references  pertaining  to  pilot 
selection,  and  3)  identify  the  knowledge(K),  skills(S),  abilities(A),  and  other(O)  characteristics 
that  are  critical  to  success  as  a  pilot.  This  report  is  concerned  with  the  third  goal;  it  reviews 
studies  that  have  identified  the  necessary  KSAOs  for  fixed-wing  and  rotary-wing  pilots  using 
ratings  by  subject  matter  experts  (SMEs)  or  factor  analysis. 

This  report  is  narrowly  focused  and  is  not  concerned  with  methodological  issues  pertaining  to 
job  analysis  or  selection  instrument  development.  It  also  is  not  concerned  with  analyses  of  flying 
training  systems.  That  is,  it  does  not  include  reports  evaluating  military  flight  training  in  terms  of 
the  knowledge  and  skills  taught  versus  those  needed  to  fly  various  military  aircraft. 

Originally,  this  report  was  to  include  studies  that  identified  the  critical  KSAOs  for  both  ab  initio 
and  experienced  civilian  pilots,  as  well  as  for  ab  initio  military  pilots.  However,  no  civilian 
studies  in  English  were  located.  Consequently,  this  reports  deals  only  with  the  KSAOs  for  ab 
initio  rotary-wing  and  fixed-wing  military  pilots.  The  KSAOs  for  unmanned  aerial  systems  are 
described  in  a  separate  report. 

Limitations 

The  studies  reviewed  in  this  report  allow  few  conclusions  about  critical  KSAOs  because  of  three 
recurring  methodological  problems.  The  first  involves  the  treatment  of  knowledge,  skills,  and 
“other”  characteristics.  The  second  concerns  the  use  of  established  taxonomies  to  identify  the 
critical  abilities.  The  third  involves  the  inclusion  of  attributes  in  taxonomies  that  are  poorly 
attested  in  the  literature  as  abilities.  Each  of  these  will  be  discussed  in  turn. 

Knowledge,  Skills,  and  Other  Attributes 

Military  ab  initio  flight  training  assumes  that  all  necessary  knowledge  and  skills  are  taught 
during  flight  training.  Consequently,  investigators  studying  critical  KSAOs  typically  are  not 
concerned  with  knowledge  and  skills.  Indeed,  only  three  studies  (Agee,  Shore,  Alley,  Barto,  & 
Halper,  2009;  Houston  &  Bmskiewics,  2006;  Youngling,  Levine,  Mochamuk,  &  Weston,  1977) 
were  located  that  examined  the  knowledge  and  skills  required  to  be  a  successful  pilot,  and 
comments  about  knowledge  and  skills  are  limited  to  these  studies. 

In  the  context  of  pilot  selection,  “other”  attributes  usually  refer  to  personality  characteristics. 

Five  studies  included  personality  traits  in  their  KSAO  lists  (Agee  et  al.,  2009;  Carretta,  Rodgers, 
&  Hansen,  1993;  Houston  &  Bruskiewicz,  2006;  McAnulty  &  Jones,  1984;  Youngling  et  al., 
1977).  Only  Agee  et  al.  provide  a  source  for  the  personality  traits  examined  in  their  study.  The 
variety  of  “personality  traits”  included  in  the  other  four  studies  suggests  that  the  authors  did  not 
obtain  their  trait  list  from  one,  established  theory  of  personality;  the  traits  appear  to  have  been 
selected  from  several  sources.  Houston  &  Bruskiewicz  (2006),  however,  did  take  a  large  number 
of  their  traits  from  the  Five  Factor  Personality  Theory.  Some  traits  appear  to  have  been  derived 
from  subject  matter  expert  (SME)  jargon  (“psychological  stress  resistance,”  “responsibility  for 
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men  in  combat”)  and  are  not  attested  to  in  the  literature  as  distinct  traits.  When  appropriate,  the 
lack  of  attestation  is  noted. 

Taxonomies 

All  of  the  studies  reviewed  were  concerned  with  abilities.  Consequently,  most  of  this  report  is 
concerned  with  identifying  the  abilities  that  are  critical  to  success  as  a  pilot.  No  one  study  can  be 
relied  on  exclusively  to  identify  those  abilities  critical  to  success  as  a  pilot.  Instead,  the  results  of 
many  studies  must  be  examined  to  identify  those  abilities  repeatedly  found  to  be  critical.  To 
make  meaningful  comparisons,  all  of  the  studies  should  use  a  common  (or  very  similar) 
framework  (taxonomy)  of  abilities.  The  scientific  literature  should  attest  to  each  ability  included 
in  the  taxonomy  as  a  distinct,  measurable  entity  that  has  been  identified  through  rigorous, 
scientific  testing.  Finally,  to  make  meaningful  comparisons,  the  investigators  should  use  the 
established  definitions  of  the  abilities  given  in  the  taxonomy.  Identifying  critical  abilities  from 
studies  with  widely  differing  definitions  of  the  same  abilities  is  problematic. 

Many  ability  taxonomies  have  been  developed  for  job  analysis  and  personnel  classification  (see 
Knapp,  Russell,  &  Campbell,  1995  for  a  discussion).  These  taxonomies  have  been  carefully 
developed  using  established  scientific  methods  and  provide  a  carefully  constructed  description  of 
each  ability.  One  of  these  taxonomies  was  developed  by  Fleishman  (see  Fleishman  &  Reilly, 

2001  for  the  most  recent  version)  and  is  used  by  most  investigators  involved  in  pilot  selection. 
Few  investigators,  however,  use  the  standard  definitions  or  restrict  themselves  to  the  abilities 
included  in  the  taxonomy. 

All  five  studies  using  ratings  or  rankings  to  identify  critical  abilities,  added  abilities  to  those  of 
Fleishman’s  taxonomy,  or  made  composites  from  two  or  more  abilities.  The  problematic  abilities 
discussed  in  the  next  section  comprise  many  of  the  additions.  None  of  the  authors  gave  any 
rationale  for  such  additions.  However,  Miller,  Eschenbrenner,  Marco,  and  Dohme  (1981)  appear 
to  have  made  the  additions  because  the  SMEs  participating  in  the  preliminary  efforts  used 
terminology  that  suggested  the  existence  of  abilities  not  included  in  the  taxonomy.  Agee  et  al. 
(2009)  specifically  stated  that  SMEs  were  allowed  to  add  attributes  that  they  felt  were  missing. 

Composite  abilities  present  a  number  of  interpretation  problems.  For  example,  different  SMEs 
may  weigh  the  components  differently,  which  may  increase  the  rating  variance  and  decrease  the 
reliability  of  the  scale.  The  different  weights  assigned  subjectively  by  the  raters  also  make 
comparisons  to  ratings  of  individual  components  problematic.  Nevertheless,  composite  abilities 
can  be  useful  in  certain  situations.  For  example,  the  study  by  Driskill,  Koonce,  Nance,  and 
Weissmuller  (2001)  was  designed  to  identify  specific  areas  that  needed  more  in-depth  research 
(Weissmuller,  personal  communication,  Jan.  4,  2011).  By  using  composite  abilities,  the 
investigators  were  able  to  identify  promising  areas  for  future  research  and  keep  the  number  of 
ratings  performed  by  each  SME  to  a  reasonable  level.  A  second  reason  for  using  composites 
occurs  when  SMEs  are  unable  to  distinguish  between  two  abilities.  Only  two  of  the  studies 
reviewed  (Driskill  et  al.,  2001;  Houston  &  Bruskiewicz,  2006)  included  composite  abilities  in 
their  attribute  list.  The  composites  will  not  be  analyzed  further  or  described  because  of  the 
above-mentioned  interpretation  problems. 
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Problematic  Abilities 

Three  abilities — Mechanical  Aptitude,  Timesharing,  and  Situational  Awareness — were 
frequently  added  to  Fleishman’s  taxonomy.  Before  the  studies  are  presented,  the  scientific 
evidence  supporting  the  existence  of  Mechanical  Aptitude,  Timesharing,  and  Situational 
Awareness  as  distinct  abilities  will  be  presented.  Differing  definitions  of  these  abilities  will  be 
discussed  at  the  same  time  because  the  identification  of  an  ability  and  its  definition  are  closely 
related. 

MECHANICAL  APTITUDE.  Several  of  the  studies  cited  in  this  report  include  Mechanical 
Aptitude  in  their  list  of  abilities.  Carroll  (1993)  discusses  problems  associated  with  Mechanical 
Aptitude  at  length.  For  the  purposes  of  this  report,  the  major  problem  with  Mechanical  Aptitude 
is  the  lack  of  a  clear  empirical  distinction  between  Mechanical  Aptitude  and  various  spatial 
abilities.  In  data  sets  containing  tests  of  both  spatial  ability  and  Mechanical  Aptitude,  Carroll  (pg. 
526)  notes  that  tests  of  Mechanical  Aptitude  may  correlate  highly  (“load  on”)  with  the 
Visualization  factor,  which  defines  one  of  three  major  spatial  abilities.  Mechanical  Aptitude  may 
also  load  on  a  factor  distinct  from  that  of  the  spatial  ability  factors  or  may  load  with  the  spatial 
ability  tests  on  one  factor  while  also  loading  on  a  separate  mechanical  factor.  A  second  problem 
noted  by  Carroll  is  that  Mechanical  Aptitude  tests  may  assess  two  different  abilities — one  that  is 
associated  with  mechanical  knowledge  (familiarity  with  terminology,  the  function  of  various 
tools,  etc.) — and  one  that  concerns  judgments  of  mechanical  operations.  Most  Mechanical 
Aptitude  tests  contain  a  mixture  of  problems;  therefore,  separating  these  two  abilities  is  difficult. 

Adding  Mechanical  Aptitude  to  Fleishman’s  taxonomy  (Fleishman  &  Reilly,  2001)  may 
introduce  confusion.  Fleishman’s  taxonomy  contains  two  spatial  abilities:  Visualization  and 
Orientation.  Studies  using  factor  analysis  to  identify  the  required  abilities  may  find  that  a  test  of 
Mechanical  Aptitude  does  not  load  on  a  distinct  factor  and/or  may  increase  the  difficulty  of 
identifying  spatial  factors.  Those  studies  using  SME  rankings  or  ratings  to  identify  critical 
abilities  may  encounter  a  different  problem.  If  Mechanical  Aptitude  consists  of  two  abilities  as 
Carroll  (1993)  suggests,  then  different  SMEs  may  think  about  Mechanical  Aptitude  differently. 
This  difference  may  increase  the  variance  of  the  ratings/rankings  of  Mechanical  Aptitude  and 
make  comparisons  with  the  ratings/rankings  of  other  abilities  questionable. 

MULTI-TASKING.  The  existence  of  some  type  of  a  multi-tasking  ability  has  been  a  matter  of 
discussion  for  over  100  years  (see  Damos  &  Wickens,  1980  for  a  discussion).  Factor  analyses 
show  weak  and  contradictory  evidence  for  such  an  ability  (Carroll,  1993).  Nevertheless,  many  of 
the  taxonomies  used  in  pilot  selection  include  some  type  of  multi-tasking  ability. 

The  definition  of  this  ability  differs  widely  between  investigators.  Fleishman’s  definition  of 
Timesharing  is  “The  ability  to  shift  back  and  forth  efficiently  between  two  or  more  activities  or 
sources  of  information.”  The  definitions  of  the  other  studies  that  included  some  type  of 
multitasking  attribute  are  shown  in  Table  1.  The  Carretta  et  al.  definition  of  Divided  Attention  is 
the  closest  to  Fleishman’s. 

The  definitions  shown  in  Table  1  differ  on  three  dimensions.  The  first  is  successive  (Carretta  et 
al.’s  Divided  Attention,  1993;  Miller  et  al.,  1981)  versus  simultaneous  processing  (Agee  et  al., 
2009;  Carretta  et  al.’s  Timesharing;  Driskill  et  al.,  2001;  Houston  &  Bruskiewicz’s  Divided 


3 


Attention,  2006)  of  input  sources.  The  second  dimension  includes  task  prioritization,  which  may 
be  excluded  from  the  definition  (Agee  et  al.,  2009;  Carretta  et  al.’s  Divided  Attention),  included 
(Driskill  et  al.,  2001;  Carretta  et  al.’s  Timesharing),  or  regarded  as  a  separate  ability  (Houston  & 
Bruskiewicz,  2006;  Miller  et  al.,  1981).  The  third  dimension  is  task  integration,  which  is 
included  only  in  the  Timesharing  definition  of  Carretta  et  al.  Given  the  differing  definitions  of 
multi-tasking  ability,  cross-study  comparisons  must  be  made  with  extreme  caution. 


Table  1 


Definitions  and  Properties  of  Multi-Tasking  Abilities  by  Study 


Study 

Name 

Definition 

Simultaneous 

Versus 

Successive 

Processing 

Prioritization 

Task 

Integration 

Agee  et  al. 
(2009) 

Task 

Management 

“Ability  to 
handle  multiple 
aspects  of  the 
mission 

simultaneously.” 

Simultaneous 

No 

No 

Miller  et  al. 
(1981) 

Divided 

Attention 

“The  ability  to 
use  information 
obtained  by 
shifting  between 
two  or  more 

sensory 

modalities.” 

Successive 

No 

No 

Miller  et  al. 
(1981) 

Establish 

Priorities 

“The  ability  to 
determine  which 
of  several 
problems  must 
be  solved  first.” 

NA 

Yes 

No 

Driskill  et  al. 
(2001) 

Multi-tasking 

“The  ability  to 

effectively 

prioritize 

workload  and 

perform 

simultaneous 

efforts  under 

demanding 

situations.” 

Simultaneous 

Yes 

No 

Houston  & 
Bruskiewicz 
(2006) 

Divided 

Attention 

“To  pay 
attention  to 
multiple  tasks 
occurring  at  the 
same  time.” 

? 

No 

No 
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Houston  & 
Bruskiewicz 
(2006) 

Cognitive 

Task 

Prioritization 

“To  properly 
pay  attention  to 
tasks  in  order  to 
achieve 

subgoals  which 
support  the 
overall  mission 
goal;  that  is, 
ensure  that  the 
pilot  is  doing 
what  he  or  she 
should  be  doing 
at  all  times.” 

? 

Yes 

No 

Carretta  et 
al.  (1993) 

Timesharing 

“The  ability  to 
observe  several 
sources  of 
information, 
actions  or  tasks 
at  the  same 
time,  to  combine 
them,  allot  task 
priorities,  and 
integrate  them 
into  actions  that 
have  to  be 
performed.” 

Simultaneous 

Yes 

Yes 

Carretta  et 
al.  (1993) 

Divided 

Attention 

“The  ability  to 
shift  back  and 
forth  between 

two  or  more 

sources  of 
information.” 

Successive 

No 

No 

SITUATIONAL  AWARENESS.  Arguably,  the  most  problematic  construct  pertaining  to  pilot 
selection  is  Situational  Awareness.  Situational  Awareness,  like  multi-tasking  abilities,  suffers 
from  a  lack  of  an  established  definition.  Perhaps  the  most  common  definition  is  Endsley’s 
(1995),  “The  perception  of  the  elements  in  the  environment,  within  a  volume  of  time  and  space, 
the  comprehension  of  their  meaning  and  the  projection  of  their  status.”  Three  studies  reviewed  in 
this  report  provide  other  definitions  of  Situational  Awareness.  Agee  et  al.  (2009)  define 
Situational  Awareness  as  “the  ability  to  track  changing  information  and  events  in  a  dynamic 
environment  and  assess  impact  on  the  mission.”  Carretta  et  al.’s  (1993)  definition  is  “the  state  of 
constant  mental  readiness  in  order  to  respond  to  situational  changes.”  Houston  &  Bruskiewicz 
(2006)  define  Situational  Awareness  as  the  ability  “to  accurately  perceive  self,  others,  and 
aircraft  in  relation  to  the  environment.”  None  of  these  three  definitions  is  as  broad  as  Endsley’s, 
and  all  differ  significantly  from  each  other. 
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The  scientific  status  of  the  Situational  Awareness  construct  is  unclear.  Carroll  (1993)  provides  no 
data  supporting  the  existence  of  Situational  Awareness  as  a  distinct  ability.  However,  given  that 
at  least  three  measures  of  Situational  Awareness  are  needed  to  identify  a  specific  factor,  the  lack 
of  factorial  evidence  for  this  attribute  is  not  surprising.  Carretta,  Perry,  and  Ree  (1996)  assumed 
that  Situational  Awareness  is  a  skill  and  attempted  to  predict  ratings  of  Situational 
Awareness  of  F-15  pilots  by  peers  and  supervisors  using  flying  hours,  personality,  general 
cognitive  ability,  and  measures  of  psychomotor  ability.  Flying  hours  demonstrated  the  highest 
correlation  with  ratings,  followed  by  general  cognitive  ability  (visual  short-term  memory,  spatial 
short-term  memory,  spatial  reasoning,  and  divided  attention).  This  finding  supports  the  notion 
that  Situational  Awareness  is  a  skill,  not  an  ability,  that  can  be  improved  by  practice.  Endsley 
and  Bolstad  (1994)  correlated  scores  on  a  measure  of  Situational  Awareness  with  measures  from 
a  battery  consisting  of  biographical  data,  and  spatial,  perceptual,  and  timesharing  tests.  The  study 
had  a  small  sample  (14  to  21  pilots  per  measure),  and  only  the  correlation  with  single-task 
tracking  performance  appeared  to  be  statistically  significant.  Again,  this  result  points  to 
Situational  Awareness  as  a  skill,  not  an  ability. 

Comparing  Situational  Awareness  ratings  among  Agee  et  al.  (2009),  Carretta  et  al.  (1993),  and 
Houston  &  Bruskiewicz  (2006)  is  problematic  because  of  the  differences  in  the  definitions.  The 
limited  data  available  appear  to  indicate  that  Situational  Awareness  is  a  skill  that  is  directly  tied 
to  flight  time.  If  Situational  Awareness  is  a  skill,  its  usefulness  in  an  ab  initio  selection  will 
depend  on  the  cost  of  training  individuals  to  acceptable  levels  of  proficiency  relative  to  the  cost 
of  selecting  for  high  levels  of  Situational  Awareness. 

APPROACH— SEARCH  STRATEGY 

DAS  searched  four  online  databases  to  locate  articles  for  this  report:  Ingenta, 
WorldWideScience.org,  PubMed,  and  Psychlnfo.  Ingenta  has  over  5  million  articles  available. 

No  information  was  readily  available  on  the  number  of  citations  included  in 
WorldWideScience.org,  but  any  search  conducted  on  the  site  accesses  at  least  80  databases. 
PubMed  has  more  than  20  million  citations  for  biomedical  literature.  The  Psychoinfo  database 
contains  more  than  3  million  references  and  includes  relevant  journals  such  as  Human  Factors, 
Ergonomics,  and  the  International  Journal  of  Aviation  Psychology.  DAS  searched  its  own 
database,  which  contains  over  1900  references  pertaining  to  pilot  selection.  Additionally,  all  Air 
Force  Human  Resources  Laboratory  reports  published  between  1953  and  2010  were  searched,  as 
were  all  of  the  conference  proceedings  of  the  International  Military  Testing  Association.  All 
searches  were  conducted  using  the  terms  “pilot  and  task  analysis,”  “pilot  and  job  analysis,”  and 
“pilot  job.”  The  search  was  restricted  to  articles  published  in  English. 

The  search  produced  only  nine  relevant  reports.  Several  contractors’  reports  could  not  be 
obtained  from  any  source,  including  the  Defense  Technical  Information  Center  (DTIC).  Thus, 
this  report  includes  only  five  studies  that  rated  or  ranked  the  KSAOs  to  identify  the  most  critical 
attributes,  two  that  used  factor  analysis,  and  two  that  used  unique  approaches  to  the  problem. 
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RATINGS  STUDIES 


Fixed  Wing — Ab  Initio — General 

Driskill  et  al.  (2001)  identified  the  abilities  required  of  United  States  Air  Force  (USAF)  pilot 
training  candidates.  The  authors  began  their  study  by  developing  a  task  list  for  the  Tl-A,  T-37, 
T-38,  and  JPATS  aircraft.  The  authors  identified  nine  categories  of  abilities  to  be  used  in 
evaluating  the  tasks.  All  categories  were  rated  on  a  scale  from  1  (not  at  all  needed  to  perform  the 
task  satisfactorily)  to  4  (needed  to  a  great  extent).  The  authors  indicate  that  all  nine  categories 
were  distilled  from  Fleishman’s  work  but  do  not  indicate  the  specific  abilities  included  in  the 
categories.  They  provide  definitions  for  only  four  of  the  categories — Mechanical  Aptitude, 
Selective  Attention,  Multi-Tasking,  and  Perceptual  Motor  Skills.  The  definitions  and  problems 
associated  with  Mechanical  Aptitude  and  Multi-Tasking  have  been  discussed  previously. 

Driskill  et  al.’s  definition  of  Selective  Attention  is  similar  to  that  of  Fleishman  (Fleishman  & 
Reilly,  2001),  but  their  definition  of  Perceptual  Motor  Skills  corresponds  to  none  of  Fleishman’s 
abilities.  Three  of  the  remaining  categories — Intelligence,  Perceptual  Skills,  and  Fine  Motor 
Skills — appear  to  be  composites  of  several  unidentified  abilities.  One  category,  Information 
Recall,  cannot  be  identified  with  any  one  or  group  of  Fleishman’s  abilities.  Spatial  Orientation 
also  is  not  defined. 

Instructor  pilots  and  student  pilots  who  were  currently  flying  or  who  were  familiar  with  one  of 
the  four  aircraft  were  recruited  as  raters.  Because  of  the  length  of  the  task  lists,  the  respondents 
were  divided  into  three  groups  with  each  group  rating  the  tasks  for  its  aircraft  on  a  different  set 
of  three  abilities.  The  ratings  of  the  task  lists  are  not  provided  in  the  report.  The  respondents  also 
rated  each  of  1 1  major  mission  events  common  to  all  aircraft  in  terms  of  their  importance, 
difficulty  to  learn,  and  the  extent  to  which  each  of  the  nine  KSA  categories  was  needed  to 
perform  the  major  mission  event  satisfactorily. 

The  ratings  on  each  of  the  1 1  major  mission  events  are  shown  in  Table  2  below  for  the  categories 
representing  defined,  single  abilities  and  for  Spatial  Orientation.  These  ratings  are  averaged 
across  aircraft. 


Table  2 

Average  USAF  Pilot  Ratings  of  Need  for  Four  Ability  Categories  for  Performing  Tl-A,  T-37,  T- 
38,  and  JPATS  Mission  Events  (Driskill  et  al.,  2001) 


Mission  Event 

Ability 

Spatial 

Orientation 

Selective 

Attention 

Multi-Tasking 

Mechanical 

Aptitude 

Mission  Planning 

1.86 

1.73 

3.17 

2.29 

Patterns  and 
Landing 

3.48 

2.79 

2.80 

3.46 

Contact  Airwork 

3.27 

2.35 

2.80 

3.10 

Instrument 

Airwork 

3.36 

2.31 

3.17 

3.38 

Navigation 

3.02 

2.18 

2.91 

3.24 

Formation 

3.59 

2.43 

2.94 

3.63 

Penetration  and 

3.32 

2.37 

2.97 

3.54 
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Approach 

Emergency 

Actions 

3.24 

2.96 

3.40 

3.72 

Communications 
and  Navigation 
Equipment 

2.47 

2.38 

2.69 

2.65 

Systems 

Knowledge 

2.09 

2.81 

3.19 

1.83 

Post  Flight 
Activity 

1.63 

1.76 

2.42 

1.80 

AVERAGE 

ACROSS 

EVENTS 

2.85 

2.37 

2.95 

2.97 

The  only  surprising  finding  in  this  table  concerns  Mechanical  Aptitude.  With  the  exception  of 
the  Systems  Knowledge  event.  Mechanical  Aptitude  has  either  the  highest  or  the  second  highest 
rating  of  the  four  abilities  across  all  of  the  events.  The  magnitude  of  these  ratings  for  events  such 
as  patterns  and  landings  and  penetration  and  approach  is  difficult  to  understand. 

Agee  et  al.  (2009)  were  concerned  with  revising  the  Air  Force  Officer  Qualification  Test 
(AFOQT)  to  ensure  that  the  KSAOs  assessed  in  the  test  were  the  most  relevant  for  ensuring  that 
Air  Force  officers  could  attain  “fully  qualified”  status  in  their  Air  Force  specialty,  one  of  which 
was  pilot.  Surveys  were  distributed  to  Active  Duty,  Air  Force  Reserve  Command,  and  Air 
National  Guard  pilots  using  the  internet.  The  surveys  asked  the  respondents  to  rate  the  degree  to 
which  a  given  attribute  was  characteristic  of  a  fully  qualified  pilot.  The  scale  ranged  from  0  (not 
required  to  become  qualified)  to  5  (fully  qualified  pilots  require  a  high  degree  of  this  ability). 
Behavioral  examples  were  used  to  help  define  the  scale  values  for  the  raters.  Agee  et  al.  received 
1092  useable  surveys. 

The  attributes  included  in  the  survey  initially  were  derived  from  Project  MIDAS  (Dittmar, 
Weissmuller,  Driskill,  Hand,  &  Earles,  1994).  Four  subsequent  focus  groups  added  items, 
deleted  items,  changed  the  definitions,  and  changed  the  behavioral  anchors.  The  result  was  a  list 
of  attributes  containing  27  “cognitive  abilities,”  12  “psychomotor  abilities,”  and  15 
“interpersonal  attributes.”  Four  of  the  “cognitive  abilities”  (Technology  Literacy,  Electro- 
Mechanical  Science,  Aviation  Knowledge,  and  Earth/Weather  Science)  are  knowledges  and  will 
be  discussed  separately.  Another,  Information  Processing/Sensor  Management,  appears  to  be  a 
combination  of  computer  skills  (stream  live  data. . .)  and  Timesharing  (drive  and  listen  to  GPS 
instructions).  This  ability  also  will  not  be  discussed  further.  Seven  of  the  “cognitive  abilities” 
appear  to  be  traits  or  cognitive  styles  and  will  be  classified  as  “Other.”  Six  of  the  psychomotor 
abilities  are  physical  abilities  (e.g.  visual  acuity,  static  strength,  etc.),  which  are  not  the  focus  of 
this  report. 

The  average  rating  of  the  15  cognitive  abilities,  6  “other”  characteristics,  6  psychomotor  abilities, 
the  4  knowledges,  and  the  15  interpersonal  attributes  are  shown  in  Tables  3,  4,  5,  6,  and  7, 
respectively.  The  attributes  are  grouped  into  tables  to  allow  more  meaningful  comparison  with 
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other  studies  presented  later  in  this  report.  That  is,  attributes  that  Agee  et  al.  (2009)  classified  as 
“cognitive  abilities”  may  be  placed  in  the  “other”  category  or  in  the  “knowledge”  category  to 
facilitate  comparison. 


Table  3 


Average  USAF  Pilot  Ratings  of  Cognitive  Ability  Relevance  to  Pilot  Qualification  (Agee  et  al., 
2009) _ 


Ability 

Average  Rating 

Situational  Awareness 

4.88 

Spatial  Orientation 

4.83 

Task  Management  (Multi-Tasking) 

4.74 

Memorization 

4.71 

Listening  Comprehension 

4.69 

Mathematical  Computation 

4.38 

Perceptual  Vigilance 

4.37 

Reading  Comprehension 

4.24 

Oral  Expression 

4.12 

Visualization 

4.11 

Deductive  Reasoning 

3.90 

Inductive  Reasoning 

3.86 

Pattern  Recognition 

3.62 

Mathematical  Reasoning 

3.56 

Written  Expression 

3.32 

Table  4 

Average  USAF  Pilot  Ratings  of  “Other”  Characteristic  Relevance  to  Pilot  Qualification  (Agee  et 
al.,  2009) _ 


Characteristic 

Average  Rating 

Adaptability 

4.78 

Prioritization 

4.46 

Foresight 

4.36 

Critical  Thinking 

4.14 

Planning 

4.00 

Perspective 

3.91 

Resourcefulness 

3.78 

It  should  be  noted  that  Prioritization  does  not  refer  to  a  multi-tasking  attribute.  It  is  concerned 
with  efficient  scheduling. 
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Table  5 


Average  USAF  Pilot  Ratings  of  Psychomotor  Ability  Relevance  to  Pilot  Qualification  (Agee  et 
al.,  2009) _ 


Psychomotor  Ability 

Average  Rating 

Rate  Control 

4.45 

Choice  Reaction  Time 

4.33 

Hand/Eye  Coordination 

4.31 

Finger  Dexterity 

4.22 

Multi-Limb  Coordination 

3.86 

Arm-Hand  Steadiness 

3.51 

Table  6 

Average  USAF  Pilot  Ratings  of  Knowledge  Relevance  to  Pilot  Qualification  (Agee  et  al.,  2009) 


Knowledge 

Average  Rating 

Aviation  Knowledge 

4.75 

Electro-Mechanical  Science 

3.26 

Technology  Literacy 

3.23 

Earth/  Weather  Science 

3.13 

Table  7 


Average  USAF  Pilot  Ratings  for  Interpersonal  Attribute  Relevance  to  Pilot  Qualification  (Agee 
et  al.,  2009) _ 


Attribute 

Average  Rating 

Integrity 

4.51 

Assuming  Responsibility 

4.47 

Resilience 

4.46 

Responsiveness 

4.44 

Self-Discipline 

4.35 

Decisiveness 

4.34 

Self-Assessing 

4.24 

Teaching 

4.13 

Cooperating 

4.01 

Work  Effectively  in  Uncomfortable  Situations 

3.80 

Selflessness 

3.75 

Persuading/  Influencing 

3.47 

Work  Effectively  in  Isolation  Settings 

3.43 

Mediation 

3.27 

Empathy 

2.95 

When  evaluating  these  data,  the  reader  must  note  that  the  survey  was  developed  for  Air  Force 
officers,  not  specifically  for  pilots.  Thus,  some  of  the  attributes  and  knowledges  are  more  general 
than  the  corresponding  constructs  in  studies  examining  only  pilots.  Additionally,  SMEs  were 
allowed  to  change  definitions  of  attributes  and  add  attributes.  Thus,  Fleishman’s  definition  for  a 
given  ability  may  be  very  different  from  the  definition  included  in  this  survey. 
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Tables  3  through  7  show  very  high  ratings,  with  raters  indicating  that  intermediate  to  high 
degrees  of  the  cognitive  and  psychomotor  attributes  were  necessary  to  become  fully  qualified. 

The  only  knowledge  with  a  high  rating  was  Aviation  Knowledge,  which  was  expected.  The 
“other”  attributes  generally  were  rated  highly,  whereas  only  the  interpersonal  attributes  that  are 
traditionally  associated  with  pilots  (Integrity,  Self-Discipline,  etc.)  were  rated  highly.  The  only 
exception  to  this  pattern  was  Teaching,  which  was  rated  higher  than  might  be  anticipated 
(moderate  to  intermediate  degree  needed). 


Fixed  Wing — Ab  Initio — Fighter  Track 

Carretta  et  al.  (1993)  document  a  NATO  working  group  effort  to  identify  the  abilities  and  traits 
that  are  critical  to  successful  performance  as  a  fighter  pilot.  As  part  of  this  effort,  43  fighter 
pilots  from  the  U.S.,  Canada,  and  Norway  were  asked  to  rate  the  importance  of  each  of  20 
abilities  and  7  traits  for  the  performance  of  12  major  tasks.  The  tasks  were  chosen  because  they 
were  unique  to  high-performance  fighter  aircraft  and  considered  critical  for  successful 
performance.  Each  ability  and  trait  was  rated  on  a  5-point  scale.  Although  no  description  of  the 
scale  is  given,  the  results  indicate  that  higher  numbers  were  associated  with  greater  importance. 

Although  the  authors  never  specifically  cite  sources  for  the  abilities  and  traits,  17  of  the  20 
abilities  match  those  of  Fleishman  (Fleishman  &  Reilly,  2001).  Two  of  the  three  other  abilities 
are  Situational  Awareness  and  Timesharing.  Carretta  et  al.’s  (1993)  definition  of  Timesharing 
and  its  emphasis  on  task  priorities  and  the  integration  of  information  sources  has  been  noted 
previously.  The  authors,  however,  included  a  second  multi-tasking  ability,  Divided  Attention, 
which  matches  Fleishman  and  Reilly’s  (2001)  Time  Sharing.  The  one  remaining  ability, 
Reasoning,  could  not  be  identified  and  is  not  discussed  further. 

The  seven  traits  assessed  in  this  study  are  Aggressiveness,  Risk  Taking,  Assertiveness, 

Emotional  Stability,  Cooperativeness,  Feadership,  and  Achievement  Motivation.  Emotional 
Stability  is  a  common  name  for  one  of  the  five  major  constructs  in  the  Five  Factor  personality 
theory  (McCrae  &  Costa,  1997).  Achievement  motivation  is  a  facet  of  Conscientiousness, 
another  one  of  the  five  major  constructs.  Thus,  although  no  references  are  given  for  the  source  of 
the  traits,  the  Five  Factor  personality  theory  may  have  influenced  the  choice  of  attributes  to  be 
assessed. 

SMEs  ranked  a  combined  list  of  the  27  abilities  and  traits,  i.e.  the  SMEs  did  not  rank  the  abilities 
separately  from  the  traits.  For  the  purposes  of  this  paper,  the  average  rank  of  the  traits  is 
presented  in  a  separate  table  from  the  abilities.  The  average  rank  for  each  of  the  19  identifiable 
abilities  is  given  in  Table  8;  the  average  rank  for  the  traits  is  in  Table  9.  The  data  in  these  tables 
were  generated  only  by  the  U.S.  pilots  (N=10).  The  ranks  are  shown  in  both  tables  so  that  the 
reader  may  re-integrate  the  tables  if  desired. 
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Table  8 


Average  USAF  Fighter  Pilot  Rankings  and  Ratings  of  Ability  Relevance  to  Major  Tasks 
(Carretta,  1993;  A  =10) _ 


Ability 

Average  Rank  Order 

Average  Rating 

Situational  Awareness 

2 

4.44 

Timesharing 

3 

4.43 

Memorization 

4 

4.42 

Perceptual  Speed 

6 

4.37 

Selective  Attention 

7 

4.29 

Divided  Attention 

9 

4.19 

Spatial  Orientation 

10 

4.14 

Response  Orientation 

11 

4.13 

Flexibility  of  Closure 

12 

4.12 

Information  Ordering 

13 

4.04 

Psychomotor  Coordination 

14 

3.95 

Control  Precision 

18 

3.59 

Oral  Comprehension 

20 

3.37 

Oral  Expression 

21 

3.35 

Visualization 

22 

3.21 

Written  Comprehension 

23 

3.00 

Number  Facility 

24 

2.70 

Rate  Control 

26 

1.54 

Written  Expression 

27 

1.23 

Table  9 


Average  USAF  Fighter  Pilot  Rankings  and  Ratings  of  Non-Cognitive  Trait  Relevance  to  Major 
Tasks  (Carretta,  1993;  N=  10) _ 


Trait 

Average  Rank  Order 

Average  Rating 

Achievement  Motivation 

1 

4.51 

Aggressiveness 

8 

4.20 

Cooperativeness 

15 

3.94 

Emotional  Stability 

16 

3.92 

Risk  Taking 

17 

3.89 

Assertiveness 

19 

3.58 

Leadership 

25 

2.52 

Three  entries  in  the  tables  require  comment.  As  shown  in  Table  9,  Leadership  is  ranked 
surprisingly  low,  25th  out  of  27.  Table  10  shows  that  all  three  abilities  associated  with 
psychomotor  performance — Psychomotor  Coordination,  Control  Precision,  and  Rate  Control — 
also  were  ranked  surprisingly  low  for  pilots  of  fighter  aircraft.  Finally,  the  raters  clearly  saw  a 
difference  between  Timesharing  and  Divided  Attention,  ranking  Timesharing  as  more  important 
(third  versus  ninth  for  Divided  Attention).  This  difference  could  be  attributed  to  the  inclusion  of 
task  prioritization  and  integration  into  the  definition  of  Timesharing  or  to  its  emphasis  on 
simultaneous  processing  versus  successive  processing  in  Divided  Attention. 
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Rotary  Wing — Ab  Initio 

Miller  et  al.  (1981)  were  concerned  with  developing  a  mission  track  selection  system  that  would 
track  select  each  student  aviator  for  one  of  the  Army’s  four  major  aviation  missions:  aeroscout, 
utility,  cargo,  and  attack.  Miller  et  al.  used  a  complex  process  involving  a  large  number  of 
techniques  to  identify  the  required  KSAOs.  Because  of  the  complexity  of  the  process,  it  will  be 
described  here  only  briefly. 

The  authors  began  by  identifying  the  KSAOs  that  were  necessary  for  the  successful  completion 
of  each  of  the  four  mission  types  using  a  combination  of  interviews,  focus  groups,  and 
questionnaires.  SMEs  from  all  four  mission  types  participated  in  the  interviews  and  focus  groups 
and  completed  the  questionnaires.  Next,  Miller  et  al.  (1981)  had  SMEs  rate  the  tasks  required  to 
perform  the  missions  on  frequency,  difficulty,  and  criticality.  They  then  used  the  attributes 
(abilities  and  traits)  listed  on  the  Army’s  daily  flight  evaluation  form  to  identify  nine  attributes 
(e.g.,  Stress  Tolerance,  Confidence)  that  were  common  to  all  four  mission  types  and  asked  SMEs 
to  rate  the  importance  of  each  attribute  for  the  successful  completion  of  each  task. 

The  authors  also  made  a  statement  pertaining  to  the  importance  of  each  of  the  nine  attributes 
(“Tolerance  to  workload- induced  stress  is  the  most  important  factor  in  mission  effectiveness”) 
and  constructed  a  questionnaire  of  dyad  statements.  A  new  group  of  SMEs  selected  the  statement 
that  was  most  true.  Next,  the  results  of  the  focus  groups  and  the  two  questionnaires  were 
combined  with  the  results  of  informal  interviews  to  produce  a  list  of  20  attributes.  SMEs  from 
each  of  the  four  mission  types  were  asked  to  identify  and  rank  the  five  attributes  most  important 
to  their  mission  type. 

The  different  methods  of  identifying  critical  KSAOs  resulted  in  sets  of  non-identical  critical 
attributes.  Miller  et  al.  (1981)  decided  that  using  an  established  taxonomy  that  included  most  of 
the  ranked  attributes  was  preferable  to  using  only  the  SME-identified  attributes.  The  authors 
subsequently  compared  the  attributes  ranked  as  the  most  important  to  mission  success  using 
these  different  methods  to  those  of  several  taxonomies  found  in  the  literature  and  found  that 
Mallamad,  Levine,  and  Fleishman’s  (1980)  taxonomy  was  the  most  similar.  This  taxonomy 
consists  of  29  cognitive  and  psychomotor  abilities  and  9  physical  abilities.  Subsequently,  Miller 
et  al.  excluded  eight  physical  abilities,  three  psychomotor  abilities,  and  two  cognitive  abilities 
from  Mallamad  et  al.’s  list  and  added  Stamina,  Establish  Priorities,  and  Stress  Tolerance.  The 
definition  of  Establish  Priorities  is  vague  (see  Table  1);  it  could  refer  to  a  multi-tasking  ability 
concerned  with  the  performance  of  concurrent  tasks,  or  it  could  refer  to  a  task  management 
function  concerned  with  scheduling,  e.g.  do  the  flight  paperwork  before  the  aircraft  inspection. 
Miller  et  al.  (1981)  did  not  include  Fleishman’s  Time  Sharing  (Fleishman  &  Reilly,  2001)  in  the 
list  but  did  include  Divided  Attention  (see  Table  1  for  a  definition). 

Fifty-two  aviators  were  required  to  indicate,  using  seven-point  behaviorally-anchored  rating 
scales,  the  degree  to  which  each  of  28  attributes  contributed  to  the  successful  performance  of 
each  mission  task.  The  average  ratings  were  not  collapsed  across  mission  types.  Miller  et  al. 
(1981),  however,  do  present  the  ten  most  highly  rated  attributes  by  mission  type.  A  total  of  16  of 
the  28  attributes  were  ranked  as  critical  to  one  or  more  of  the  four  mission  types.  These  16  are 
shown  below  in  Table  10. 
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Table  10 

Attributes  Rated  Most  Important  by  U.S.  Army  Aviators  (N  =  52)  (Miller  et  al.,  1981) 


Attributes 

Perceptual  Speed 

Speed  of  Closure 

Flexibility  of  Closure 

Problem  Sensitivity 

Inductive  Reasoning 

Deductive  Reasoning 

Spatial  Orientation 

Visualization 

Verbal  Expression 

Divided  Attention 

Selective  Attention 

Multi-Limb  Coordination 

Control  Precision 

Rate  Control 

Reaction  Time 

Establish  Priorities 

Perceptual  Speed  was  in  the  top  three  most  important  attributes  in  all  four  mission  types,  as  was 
Problem  Sensitivity.  Divided  Attention  was  in  the  top  four  attributes  in  three  categories  and 
eighth  in  the  fourth.  In  contrast,  Visualization  was  rated  tenth  in  three  mission  types  and  did  not 
appear  in  the  fourth  type.  Similarly,  Multi-Limb  Coordination  was  not  included  in  the  top  10 
attributes  for  two  mission  types  and  was  rated  sixth  and  fourth  by  two  others.  Interestingly,  both 
Stamina  and  Stress  Tolerance,  which  were  added  to  the  taxonomy,  were  never  included  in  the 
top  ten  “abilities”  for  any  mission  type.  Establish  Priorities,  which  was  also  added,  was  included 
in  the  top  ten  only  once. 

Houston  &  Bruskiewicz  (2006)  performed  a  job  analysis  as  part  of  an  effort  to  develop  a  new 
selection  battery  for  Army  rotary- wing  aviators.  The  authors  began  by  collecting  information  on 
the  job  and  reviewing  the  literature.  They  then  constructed  a  preliminary  list  of  all  of  the  tasks 
performed  by  an  Army  aviator  and  a  KSAO  list.  The  KSAO  list  was  developed  from  a  number  of 
sources,  which  are  not  identified  in  the  report.  The  preliminary  KSAO  list  subsequently  was 
reviewed  for  completeness  by  a  small  group  of  SMEs.  Based  on  SME  comments,  the  authors 
shortened  both  the  task  and  the  KSAO  list  and  revised  the  lists  to  be  consistent  with  Army 
aviation  terminology.  The  authors  then  met  with  four  groups  of  SMEs,  who  reviewed  the  task  list 
and  each  task  statement  comprising  the  list.  The  task  list  and  the  task  statements  were  revised 
again  based  on  the  SME  comments. 

The  final  task  list  was  distributed  to  flight  instructors  at  Fort  Rucker.  Approximately  72%  were 
returned  (234  surveys  with  212  useable).  Each  flight  instructor  was  asked  to  rate  the  importance 
of  each  of  the  101  tasks  on  a  5-point  scale  (1  to  5).  The  scale  included  a  “0”  that  was  used  if  the 
task  was  not  part  of  a  rotary-wing  pilot’s  job.  The  instructors  also  were  asked  to  rate  the 
importance  of  12  knowledge  categories,  6  skills,  26  abilities,  and  48  other  characteristics  using  a 


14 


5-point  scale.  The  lowest  score  (0)  corresponded  to  “not  important.”  Scores  of  1,  2,  3,  and  4 
corresponded  to  “somewhat  important,”  “important,”  “very  important”  and  “critical,” 
respectively. 

Houston  &  Bruskiewicz  (2006)  provide  average  ratings  for  each  of  the  KSAOs.  All  six  of  the 
rated  skills  pertained  to  operation  of  the  aircraft  or  the  aircraft  systems,  such  as  “operation  of 
sensor/tracking  systems  and  equipment.”  Because  these  skills  only  can  be  acquired  through 
helicopter  training,  they  will  not  be  discussed.  Most  of  the  12  knowledge  categories  also  were 
concerned  with  information  that  could  only  be  acquired  through  military  flight  training. 
However,  four  categories  represented  information  that  a  motivated  ab  initio  pilot  could  be 
expected  to  know.  These  are  shown  below  in  Table  11. 

Table  11 


Average  Army  Flight  Instructor  Importance  Ratings  of  Knowledge  Categories  for  Rotary-Wing 
Pilots  (N  =  212)  (Houston  &  Bruskiewicz,  2006) 


Category 

Rating 

Flight  Rules  and  Regulations 

3.26 

Meteorology 

2.96 

Aeronautical  Terminology 

2.89 

Aviation  Principles 

2.79 

Although  the  sources  for  abilities  and  “other”  characteristics  for  the  initial  KSAO  list  are  never 
described,  Bruskiewicz,  (personal  communication,  Sept.  29,  2010)  indicated  that  some  of  the 
abilities  and  other  characteristics  were  derived  from  other  projects  the  company  had  performed. 
Seven  of  the  26  abilities  can  be  identified  with  those  of  Fleishman  (Fleishman  &  Reilly,  2001). 
Another,  Spatial  Visualization  and  Orientation,  was  a  composite  ability,  i.e.  the  respondents 
rated  the  importance  of  both  abilities  on  one  scale,  and  will  not  be  discussed  further.  Ratings  for 
the  seven  abilities  are  shown  in  Table  12. 

Table  12 


Army  Flight  Instructor  Average  Importance  Ratings  of  Fleishman’s  Abilities  for  Rotary-Wing 
Pilots  (N  =212)  (Houston  &  Bruskiewicz,  2006) 


Ability 

Rating 

Multi-Limb  Coordination 

3.49 

Control  Precision 

3.31 

Perceptual  Speed 

3.29 

Selective  Attention 

3.28 

Oral  Comprehension 

3.25 

Simple  Reaction  Time 

3.19 

Rate  Control 

3.14 

Ratings  for  the  other  18  abilities  are  shown  in  Table  13.  Several  of  these  abilities,  such  as 
Written  Communication,  were  similar  to  those  of  Fleishman  (Fleishman  &  Reilly,  2001)  but 
were  defined  more  narrowly.  Others,  such  as  Working  Memory  and  Long-Term  Memory,  are 
recognized  in  newer  ability  models  (See  Carroll,  1993  for  a  discussion),  which  are  not  strictly 
compatible  with  taxonomies  like  those  of  Fleishman.  Other  constructs  like  Cognitive  Task 
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Prioritization,  Planning,  and  Vigilance  are  often  used  by  pilots  to  describe  cockpit  behaviors  but 
are  not  included  either  in  taxonomies  or  in  the  newer  ability  models.  Indeed,  few  data  exist 
supporting  Task  Prioritization  or  Vigilance  as  distinct  abilities;  Planning  and  Reading 
Comprehension  are  weakly  supported  as  distinct  abilities  (Carroll,  1993).  None  of  these  four 
abilities  is  included  in  Fleishman’s  taxonomy.  As  noted  in  the  Multi-Tasking  Section,  Houston  & 
Bruskiewicz  (2006)  included  a  Task  Prioritization  ability  that  is  distinct  from  their  Divided 
Attention  ability. 

Table  13 


Army  Flight  Instructor  Average  Importance  Ratings  of  Non-Fleishman  Abilities  for  Rotary- 
Wing  Pilots  (N  =  212)  (Houston  &  Bruskiewicz,  2006) 


Ability 

Rating 

Situational  Awareness 

3.76 

Divided  Attention 

3.43 

Judgment/  Decision  Making/  Problem  Solving 

3.41 

Vigilance 

3.35 

Choice  Reaction  Time 

3.34 

Oral  Communication 

3.19 

Task  Prioritization 

3.18 

Working  Memory 

3.16 

Organization/  Time  Management 

3.15 

Planning 

3.13 

Learning 

3.10 

Long-Term  Memory 

3.08 

Analytical  Ability 

3.03 

Time  Estimation 

2.91 

Reading  Comprehension 

2.85 

Mechanical  Comprehension 

2.75 

Mathematical  Ability 

2.37 

Written  Communication 

2.18 

The  most  salient  feature  of  these  two  tables  is  the  apparent  magnitude  of  the  ratings,  which 
ranged  from  3.76  to  2.18.  Only  five  abilities  (19.2%)  received  an  average  rating  below  3.00 
(very  important).  The  most  highly  rated  ability  was  Situational  Awareness  with  an  overall  rating 
of  3.76.  The  problems  associated  with  defining  and  measuring  Situational  Awareness  were 
discussed  earlier. 

The  48  attributes  in  the  “other”  category  include  both  personality  traits  and  attitudes.  This 
category  shows  a  different  pattern  of  results  from  the  ability  category.  The  ratings  ranged  from 
1.67  (Dominance)  to  3.39  (Teamwork)  with  33  attributes  (68.8%)  receiving  a  score  less  than 
3.00.  Thus,  attributes  in  the  “other”  category  were  rated  lower  than  those  in  the  ability  category 
but  still  seen  as  important. 

Again,  no  source  is  given  for  the  personality  traits,  but  they  appear  to  be  derived  from  the  Five 
Factor  personality  theory  (Goldberg,  1993;  McCrae  &  Costa,  1989,  1997).  Each  of  the  five 
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personality  constructs  reflects  the  common  variance  among  a  number  of  specific  traits,  which  are 
usually  referred  to  as  “facets.”  For  pilot  selection,  Conscientiousness  and  Neuroticism  are 
considered  to  be  two  of  the  most  important  constructs.  Conscientiousness  is  often  postulated  to 
reflect  six  facets:  Achievement  Striving,  Dutifulness,  Self-Discipline,  Order,  Competence,  and 
Deliberation  (McCrae  &  Costa,  1997).  Five  of  these  six  facets  with  their  ratings  were  assessed 
by  Houston  &  Bruskiewicz  (2006)  and  received  the  following  ratings:  Achievement  Striving 
(2.92),  Self-Discipline  (3.21),  Order  (2.27),  Competence  (3.13),  and  Deliberation  (2.83). 
Dutifulness  may  have  been  assessed  by  Dependability  (3.19).  Thus,  these  facets  were  seen  as 
“very  important”  with  the  exception  of  Order,  which  was  considered  to  be  “important.” 

All  of  the  six  facets  of  Neuroticism  also  were  assessed:  Lack  of  Anxiety  (2.79),  Lack  of  Angry 
Hostility  (2.71),  Lack  of  Depression  (2.77),  Lack  of  Self-Consciousness  (2.80),  Lack  of 
Impulsiveness  (2.56),  and  Lack  of  Vulnerability  (3.05).  These  facets  generally  were  seen  as  less 
important  than  those  associated  with  Conscientiousness  and  only  were  rated  as  “important.” 
Three  other  attributes — Stress  Tolerance,  Risk  Tolerance,  and  Leadership —  are  often  considered 
to  be  important  characteristics  of  successful  pilots  (see  Youngling  et  al.,  1977,  for  a  discussion). 
Stress  Tolerance  received  a  high  rating,  3.24.  In  contrast,  Risk  Tolerance  received  a  lower  rating, 
2.77.  Four  leadership  attributes  were  assessed,  which  were  concerned  with  delegation,  goals, 
performance  management,  and  resolving  conflicts.  The  ratings  for  these  four  attributes  were 
2.32,  2.52,  2.76,  and  2.76,  respectively.  Thus,  neither  Risk  Tolerance  nor  the  four  dimensions  of 
leadership  were  considered  “very  important”  by  the  average  rater. 
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FACTOR  ANALYSIS 


Fixed  Wing — Ab  Initio 

Fleishman  and  Ornstein  (1960)  identified  the  abilities  underlying  success  in  primary  flight 
training  in  the  USAF.  Data  were  obtained  from  63  successful  students  on  24  non-acrobatic 
maneuvers  taught  during  primary  flight  training.  Each  maneuver  was  broken  down  into  a  number 
of  elements  that  could  be  scored  as  correct  or  incorrect.  The  sum  of  the  incorrect  elements  was 
recorded  as  the  maneuver  score.  The  authors  analyzed  the  sum  of  the  first  four  presentations  of 
each  maneuver  during  any  given  flight. 

The  authors  performed  an  orthogonal  rotation  to  simple  structure  and  identified  six  factors.  The 
authors  initially  attempted  to  interpret  the  factors  by  noting  the  types  of  control  movements  or 
task  requirements  that  were  common  to  the  maneuvers  loading  on  each  factor.  This  approach, 
however,  did  not  produce  any  meaningful  interpretations.  The  authors  subsequently  interpreted 
the  factors  in  terms  of  basic  abilities,  which  did  provide  more  meaningful  interpretations  of  the 
factors.  The  six  factors  identified  were  Fine  Control  Sensitivity,  Spatial  Orientation,  Multi-Limb 
Coordination,  Response  Orientation,  and  Rate  Control.  The  sixth  factor  could  not  be  clearly 
identified  with  any  basic  ability  and  tentatively  was  labeled  Kinesthetic  Discrimination. 

Rotary  Wing — Ab  Initio 

McAnulty  and  Jones  (1984)  examined  only  the  abilities  required  during  the  instrument  phase  of 
Army  helicopter  training.  The  authors  selected  27  abilities  from  Fleishman’s  work  on  pilot 
classification  (Myers,  Jennings,  Schemmer,  &  Fleishman,  1982)  and  added  five  others  that  they 
assumed  were  important  for  successful  performance  in  this  phase  of  flight  training.  Two  of  these 
additions.  Oral  Comprehension  and  Oral  Expression,  are  in  Fleishman  and  Reilly’s  (2001)  list  of 
abilities.  However,  without  definitions,  the  relation  between  these  two  abilities  and  those  of 
Fleishman  and  Reilly  cannot  be  determined.  Another,  Stress  Tolerance,  may  be  a  personality 
characteristic  rather  than  an  ability.  The  remaining  two,  Decision  Making  and 
Movement/Position  Memory,  are  not  listed  in  Fleishman  and  Reilly  and  have  little  support  from 
Carroll  (1993)  as  distinct  abilities. 

The  authors  developed  a  list  of  16  tasks  that  students  had  to  learn  during  the  instrument  phase  of 
training.  Fifteen  SMEs  rated  each  task  on  each  of  the  32  abilities.  Because  this  report  is  a  short 
proceedings  paper,  the  authors  provide  no  definitions  of  the  abilities  or  average  ratings  by  the 
SMEs.  The  main  analysis  of  interest  is  a  maximum-likelihood  factor  analysis  with  varimax 
rotation  conducted  on  the  rating  of  the  abilities.  The  resulting  factor  analysis  showed  a  seven- 
factor  solution:  Gross  Motor  Coordination,  Fine  Motor  Coordination,  Closure/Selective 
Attention,  Visual/Perceptual  Speed,  Information  Processing,  Originality,  and  Language.  Stress 
Tolerance  and  Movement/Position  Memory  both  loaded  on  only  the  Gross  Motor  factor. 

Decision  Making  loaded  only  on  the  Information  Processing  factor.  The  seven  factors  accounted 
for  about  59%  of  the  variance. 
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OTHER  APPROACHES 


Fixed  Wing — Ab  initio — General 

Meyer,  Laveson,  Weissman,  and  Eddowes  (1974a,  1974b)  used  a  behavioral  description 
approach  (see  E.  A.  Fleishman  &  Quaintance,  1984  for  a  general  description  of  this  approach)  to 
identify  the  KSAOs  required  by  flight  training.  Each  basic  flight  maneuver  was  analyzed 
according  to  the  stimuli  required  to  cue  the  pilot  to  begin  the  maneuver,  the  “mental  actions” 
required  to  perform  the  maneuver,  and  the  motor  responses  required  to  execute  the  maneuver. 
Each  of  these  major  dimensions  was  defined  further  along  several  subdimensions.  For  example, 
the  motor  response  dimension  was  defined  in  terms  of  the  number  and  specific  controls 
(elevators,  ailerons,  etc.)  required  to  perform  the  maneuver  and  the  desired  final  state  of  the 
aircraft.  The  mental  actions  dimension  was  subdivided  into  three  subdimensions:  complexity, 
memory  accessed  (short-term  versus  long-term),  and  information  processing  (recall  of  facts 
versus  procedures).  This  method  was  “validated”  by  having  a  group  of  nine  pilots  use  the 
taxonomy  to  describe  a  set  of  basic  maneuvers.  A  high  level  of  agreement  was  found  among  the 
pilots.  Investigators  subsequently  used  this  approach  to  analyze  34  visual  maneuvers  for  the  T-37 
and  T-38  aircraft. 

Using  the  results  of  this  study  to  identify  the  required  KSAOs  is  problematic  for  two  reasons. 
First,  it  used  a  unique  taxonomy.  Indeed,  only  two  subdimensions — memory  accessed  and  recall 
of  facts  versus  procedures — approach  the  current  understanding  of  an  ability.  Second,  SMEs  did 
not  actually  rank  or  rate  the  abilities  in  terms  of  their  importance  to  the  performance  of  each 
maneuver. 

Fixed  Wing — Ab  Initio — Fighter  Track 

Youngling  et  al.  (1977)  were  concerned  with  identifying  student  pilots  who  could  become  good 
fighter  pilots.  Unlike  the  other  studies  included  in  this  report,  Youngling  et  al.  used  a 
combination  of  interviews,  questionnaires,  literature  review,  and  analyses  of  successful  pilot 
selection  systems  to  identify  the  required  KSAOs  rather  than  have  the  respondents  rate  or  rank 
the  KSAOs.  The  literature  review  and  the  analysis  of  pilot  selection  systems  were  particularly 
comprehensive  and  included  information  from  numerous  foreign  countries.  The  process  of 
identifying  the  KSAOs  through  interviews  and  questionnaires  was  conducted  in  several  stages, 
with  the  results  of  early  interviews  used  to  develop  items  for  later  questionnaires. 

The  results  of  all  of  the  sources,  including  the  literature  review  and  analyses  of  successful  pilot 
selection  systems,  were  combined  into  a  list  of  5 1  “characteristics  and  critical  skills”  that  were 
considered  necessary  for  an  effective  fighter  pilot.  For  the  purposes  of  this  report,  17  of  the 
attributes  were  eliminated  from  further  consideration  because  they  either  were  medical  factors, 
were  listed  in  tables  but  never  described  in  the  report,  or  were  not  KSAOs. 

Only  one  knowledge  category — “Equipment  Knowledge” — was  identified  as  a  critical 
knowledge.  It  refers  to  aeronautical  knowledge  and  technical  vocabulary.  Two  critical  skill 
categories  were  identified.  The  first  was  Flight  Skill,  which  refers  to  both  skill  in  training  and 
skill  in  flying  fighters.  The  second  skill  was  Aerial  Gunnery. 

Interpreting  the  remaining  34  attributes  is  problematic  because  some  are  components  of  others. 
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For  example,  one  of  the  “skills”  was  “instrument  reading,”  which  refers  to  scores  on  the 
Instrument  Comprehension  Test  of  the  AFOQT.  One  of  the  “aptitudes”  listed  was  “pilot 
composite,”  which  is  calculated  from  scores  on  the  AFOQT  including  the  Instrument 
Comprehension  Test  (Carretta  &  Ree,  1996).  The  abilities  assessed  by  the  tests  comprising  the 
pilot  composite  of  the  AFOQT  are  shown  in  Table  14  as  separate  entries.  The  personality 
attributes  and  other  attributes  also  are  shown  in  this  table. 

Table  14 


Attributes  Considered  as  Necessary  for  Success  as  a  Fighter  Pilot  (Youngling  et  al.,  1977) 


Personality  and  Other  Attributes 

Ability  Attributes 

Risk  Taking 

Motor  Coordination 

Performance  Under  Stress 

Spatial  Orientation 

Emotional  Control 

Spatial  Perception 

Ability  to  Withstand  Psychological  Stress 

Perceptual  Speed 

Anxiety  Tolerance 

Numerical 

Aggressiveness 

Verbal 

Confidence 

Mechanical 

Consideration  for  Others 

Selective  Attention 

Personality  Style 

Decision  Time 

Courage 

Alertness 

Responsibility  for  Men  in  Combat 

Physical  and  Combat  Leadership 

Teamwork 

Sociability 

Group  Loyalty 

Determination/  Desire 

Self-Discipline 

Satisfaction 

Aviation  Information 
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DISCUSSION 


General 

Some  readers  may  be  concerned  with  the  small  number  of  studies  that  have  sought  to  identify  the 
KSAOs  required  for  success  as  a  pilot.  A  larger  number  of  studies  would  have  allowed  more 
comparisons  and  would  have  provided  a  stronger  basis  for  making  recommendations.  A  larger 
database  also  could  have  been  examined  for  trends,  such  as  changes  in  KSAOs  with  changes  in 
aircraft  technology.  The  conclusions  that  can  be  drawn  from  the  studies  are  more  limited  by 
methodological  problems  associated  with  the  construction  of  the  taxonomies  used  to  identify  the 
required  KSAOs  than  by  the  small  number  of  studies.  Two  of  the  nine  studies  reviewed  did  not 
use  some  version  of  Fleishman’s  (Fleishman  &  Reilly,  2001)  taxonomy.  One  of  these  two, 

Meyer  et  al.  (1974),  used  a  unique  taxonomy  that  appears  never  to  have  been  used  again  for  pilot 
selection.  The  second,  Youngling  et  al.  (1977),  did  not  use  any  identifiable  taxonomy.  All  five 
studies  that  used  ratings  or  rankings  (Agee  et  al.,  2009;  Carretta  et  al.,  1993;  Driskill  et  al.,  2001; 
Houston  &  Bruskiewicz,  2006;  Miller  et  al.,  1981)  modified  Fleishman’s  taxonomy  by  adding 
abilities  to  the  list,  changing  definitions,  or  creating  composite  abilities. 

None  of  the  studies  making  additions  to  the  list  appear  to  have  identified  the  missing  attributes  in 
a  systematic  manner,  e.g.  by  consulting  other  rigorously  developed  taxonomies  or  compendiums 
like  Carroll  (1993)  or  identifying  the  causes  of  failures  in  flight  training.  Instead,  most  of  the 
additions  appear  to  reflect  input  from  SMEs  obtained  during  focus  groups  and  surveys.  SMEs 
may  have  used  jargon  to  describe  attributes  that  were  not  included  in  Fleishman’s  taxonomy 
(Fleishman  &  Reilly,  2001),  and  the  investigators  felt  the  need  to  reflect  the  jargon  in  the  ability 
lists.  In  at  least  one  case  (Bruskiewicz,  personal  communication,  Sept.  29,  2010),  the 
investigators  themselves  felt  the  taxonomy  was  incomplete  and  needed  additional  attributes.  As 
noted  earlier,  several  common  additions  to  the  ability  list  either  are  not  attested  as  distinct 
abilities  (Situational  Awareness)  or  have  weak  and  conflicting  evidence  for  their  existence 
(Timesharing  and  Mechanical  Aptitude). 

Changes  to  Fleishman’s  definitions  also  cause  problems  in  interpreting  the  results.  As  described 
earlier,  the  most  extreme  changes  concerned  Fleishman’s  Timesharing.  Not  only  was  the 
definition  changed,  but  new  multi-tasking  abilities  that  are  not  attested  in  the  literature  also  were 
added  to  the  attribute  list.  Endsley’s  (1995)  definition  of  Situational  Awareness  also  was  altered 
significantly.  Although  Timesharing  and  Situational  Awareness  are  the  most  extreme  examples 
of  varying  definitions,  five  sets  of  investigators  using  the  ratings  and  rankings  approach  changed 
many  of  the  ability  definitions  in  less  dramatic  ways.  How  these  changes  affected  the  ratings  and 
rankings  is  unknown,  and  cross-study  comparisons  must  be  made  very  carefully. 

With  these  serious  caveats,  few  general  statements  can  be  made  about  the  attributes  required  for 
success  as  a  pilot.  Historically,  several  abilities  have  played  prominent  roles  in  pilot  selection 
batteries:  Spatial  Orientation,  Perceptual  Speed,  Numerical/Quantitative  Ability,  Timesharing, 
Mechanical  Aptitude,  and  Multi-Limb  Coordination.  These  will  be  described  below  in  order. 
Selective  Attention  was  included  in  four  of  the  task  analysis  studies  and  also  will  be  discussed, 
as  will  Situational  Awareness.  Short  sections  dealing  with  knowledge  categories  and  with  traits 
follow  the  discussion  of  abilities. 
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Specific  Abilities 


SPATIAL  ORIENTATION.  Spatial  Orientation  was  examined  by  five  studies.  Three  studies 
had  ratings  from  “critical”  (Youngling  et  al.,  1977)  to  a  ranking  of  10  out  of  27  (Carretta  et  al., 
1993)  to  inclusion  in  the  group  of  most  highly  rated  across  mission  types  (Miller  et  al.,  1981). 
Both  Agee  et  al.  (2009)  and  Driskill  et  al.,  (2001)  found  that  Spatial  Orientation  received 
relatively  high  ratings  for  flying  activities:  4.83  out  of  5.0  for  Agee  et  al.  and  3+  out  of  5  for 
Driskill  et  al. 

It  also  was  identified  in  Fleishman  and  Omstein’s  (1960)  factor  analysis. 

PERCEPTUAL  SPEED.  Perceptual  Speed  was  rated  surprisingly  highly  in  all  four  of  the 
studies  in  which  it  was  included.  Youngling  et  al.  (1977)  found  that  it  was  a  “critical”  ability. 
Carretta  et 

al.’s  (1993)  SMEs  gave  it  an  overall  rank  of  6  out  of  27.  It  was  the  most  highly  rated  ability  in 
Miller  et  al.  (1981)  and  received  a  “very  important”  rating  in  Houston  &  Bruskiewicz  (2006).  It 
also  helped  define  the  Visualization/Perceptual  Speed  factor  of  McAnulty  and  Jones  (1984). 

NUMERICAL/QUANTITATIVE.  Numerical/quantitative  attributes  were  included  in  five 
studies. 

Agee  et  al.  (2009)  included  two  attributes.  Mathematical  Computation  and  Mathematical 
Reasoning,  in  their  study.  The  definitions  of  both  of  these  attributes  corresponded  relatively 
closely  with  Fleishman’s  Number  Facility  and  Mathematical  Reasoning  (Fleishman  &  Reilly, 
2001).  Mathematical  Computation  was  highly  rated,  4.38  out  of  5.0.  Mathematical  Reasoning 
received  a  lower  rating,  3.56  out  of  5.0.  Numerical  skills  were  considered  to  be  “critical”  by  the 
fighter  pilots  in  Youngling  et  al.  (1977)  for  success  as  a  fighter  pilot.  However,  the  fighter  pilot 
raters  in  Carretta  et  al.  (1993)  ranked  Number  Facility  24th  out  of  27  abilities.  In  this  study, 
Number  Facility  was  defined  as  the  degree  to  which  adding,  subtracting,  multiplying,  and 
dividing  can  be  done  quickly  and  correctly.  Houston  &  Bruskiewicz’s  (2006)  Mathematical 
Ability,  which  they  defined  as  “to  understand  and  apply  basic  (e.g.,  addition,  rounding)  and 
advanced  (e.g.  algebra)  math  principles;  arithmetic  reasoning,”  was  considered  “important.” 
Miller  et  al.  (1981)  defined  Number  Facility  as  “the  ability  to  manipulate  numbers  in  numerical 
operations,  for  example,  adding,  subtracting  ,  multiplying,  dividing  ,  integrating  ,  etc.  The  ability 
involves  both  the  speed  and  accuracy  of  computation.”  This  attribute  was  not  rated  in  the  ten 
most  important  attributes  for  any  of  the  four  rotary-wing  missions. 

The  cause  of  the  difference  between  Youngling  et  al.’s  (1977)  rating  and  Carretta  et  al.’s  (1993) 
cannot  be  determined  because  Youngling  et  al.  did  not  provide  a  definition  of  Numerical  Skills. 
Generally,  attributes  with  definitions  similar  to  Fleishman’s  Number  Facility  (the  ability  to 
perform  basic  numerical  operations)  (Fleishman  &  Reilly,  2001)  received  lower  ratings  than 
those  emphasizing  mathematical  reasoning.  Thus,  Miller  et  al.  (1981),  Carretta  et  al.  and  Agee  et 
al.’s  (2009)  Mathematical  Computation  received  relatively  low  ratings.  In  contrast,  Agee  et  al.’s 
Mathematical  Reasoning  and  Houston  &  Bruskiewicz’s  (2006)  Mathematical  Ability  were  more 
highly  rated. 
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MULTI-TASKING.  Ratings  and  rankings  of  multi-tasking  attributes  show  a  consistent  picture 
of  highly  rated  abilities.  Agee  et  al.’s  (2009)  raters  gave  Timesharing  a  4.74  out  of  a  maximum 
of 

5.0.  In  Driskill  et  al.  (2001),  Multi-Tasking,  which  included  task  prioritization  and  simultaneous 
processing,  received  ratings  indicating  it  was  the  most  important  of  the  four  abilities  compared 
for  4  of  the  1 1  major  mission  events.  Carretta  et  al.’s  (1993)  Timesharing  included  simultaneous 
processing,  task  prioritization,  and  task  integration.  It  received  a  high  ranking,  3  out  of  27. 

Their  Divided  Attention  was  ranked  9  out  of  27.  Both  Establish  Priorities  and  Divided 
Attention,  which  emphasized  successive  processing,  were  highly  rated  by  the  pilots  in  Miller  et 
al.  (1981).  Houston  &  Bruskiewicz’s  (2006)  Divided  Attention,  which  seems  to  emphasize 
simultaneous  processing,  received  a  “very  important”  rating  (3.43).  Their  Task  Prioritization  also 
received  a  “very  important”  rating  (3.18). 

The  most  striking  feature  of  these  results  is  the  consistently  high  ratings  obtained  by  the  multi¬ 
tasking  attributes.  The  definitions  of  multi-tasking  differed  on  the  nature  of  the  processing 
(successive  versus  simultaneous)  and  the  inclusion  of  two  component  abilities:  Task 
Prioritization  and  Task  Integration  (see  Table  1).  These  differences  had  no  apparent  effect  on  the 
ratings  and,  perhaps  more  importantly,  the  related  attributes  (Establish  Priorities  and  Task 
Prioritization)  also  received  high  ratings.  This  is  particularly  surprising  given  the  weak 
attestation  in  the  literature  for  a  Timesharing  ability  and  the  complete  lack  of  attestation  for  other 
multi-tasking  abilities,  such  as  Task  Prioritization.  The  overall  impression  here  is  one  of  a  cluster 
of  attributes  related  to  multi-tasking  that  are  perceived  as  important  to  both  rotary- wing  and 
fixed-wing  pilots. 

MECHANICAL  APTITUDE.  Three  task  analysis  studies  included  Mechanical  Aptitude  in 
their  attribute  list  (Driskill  et  al.,  2001;  Youngling  et  al.,  1977;  Houston  &  Bruskiewicz,  2006). 

In  Driskill  et  al.  (2001)  it  was  consistently  rated  as  one  of  the  most  important  abilities.  Youngling 
et  al.’s  SMEs  found  that  it  was  “critical”  for  success  as  a  fighter  pilot.  Houston  and 
Bruskiewicz’s  SMEs  rated  it  as  “important.”  Thus,  despite  academic  problems  with  the  identity 
of  a  mechanical  aptitude,  pilot  raters  consider  this  an  important  attribute. 

MULTI-LIMB  COORDINATION.  Multi-Limb  Coordination  might  be  assumed  to  be  a  highly 
valued  attribute  for  rotary- wing  pilots.  However,  the  task  analysis  studies  for  rotary- wing  pilots 
showed  mixed  results.  Multi-Limb  Coordination  was  ranked  highly  for  helicopter  pilots  in 
Houston  &  Bruskiewicz  (2006)  but  not  in  Miller  et  al.  (1981).  Although  a  similar  assumption 
could  be  made  for  its  importance  in  flying  fighter  aircraft,  Multi-Limb  Coordination  was  ranked 
14th  out  of  27  attributes  for  fighter  pilots  by  Carretta  et  al.’s  (1993)  SMEs.  In  Agee  et  al.  (2009), 
it  received  a  rating  of  only  3.86  out  of  a  maximum  of  5.0.  It  loaded  only  on  the  gross 
psychomotor  factor  in  McAnulty  and  Jones  (1984). 

The  differences  between  Houston  &  Bruskiewicz  (2006)  and  Miller  et  al.  (1981)  may  be 
attributable  to  differences  in  the  importance  of  Multi-Limb  Coordination  among  the  four  Army 
mission  types:  scout,  utility,  cargo,  and  attack.  Multi-Limb  Coordination  was  rated  highly  (in  the 
top  ten  most  important  attributes)  only  for  the  attack  and  utility  missions  in  Miller  et  al.  The 
same  pattern  was  true  for  Houston  and  Bruskiewicz  although  the  overall  (average  across  mission 
types)  rating  was  3.49  out  of  4.0  (“very  important”).  Thus,  the  apparent  differences  between 
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these  two  studies  may  be  attributed  to  differences  in  the  way  the  data  were  averaged.  When  the 
data  are  examined  by  mission  type,  the  results  from  the  two  studies  basically  agree:  Multi-Limb 
coordination  is  important  for  attack  and  utility  missions.  No  explanation  can  be  offered  for  the 
average  ranking  given  by  Carretta  et  al.’s  (1993)  fighter  pilot  raters  or  for  the  relatively  low 
rating  in  Agee  et  al.  (2009). 

SELECTIVE  ATTENTION.  Selective  Attention  was  examined  in  four  of  the  rating  studies. 
Miller  et  al.  (1981)  found  that  it  was  ranked  in  the  top  10  critical  attributes  for  all  four  Army 
mission  types.  Its  ratings  were  relatively  low  in  Dri skill  et  al.  (2001)  (2+  out  of  5),  indicating  that 
the  respondents  felt  that  this  ability  was  not  greatly  needed  to  perform  major  mission  events 
successfully.  It  was  considered  to  be  “critical”  by  Youngling  et  al.  (1977).  Carretta  et  al.  (1993) 
found  that  its  average  rank  was  7  out  of  27,  indicating  that  their  respondents,  who  were  all 
fighter  pilots,  considered  it  important.  Thus,  three  of  the  four  studies  consider  Selective  Attention 
to  be  an  important  attribute  for  success  as  a  pilot. 

SITUATIONAL  AWARENESS.  Agee  et  al.  (2009),  Houston  &  Bruskiewicz  (2006),  and 
Carretta  et  al.  (1993)  included  Situational  Awareness  in  their  list  of  attributes.  Situational 
Awareness  received  the  second  highest  average  rank  in  Carretta  et  al.  and  the  highest  average 
rating  in  Agee  et  al.  and  Houston  and  Bruskiewicz.  However,  the  three  definitions  differ 
significantly  from  each  other  (see  the  Situational  Awareness  Section).  That  of  Agee  et  al.  is 
reasonably  close  to  Endsley’s  (1995)  definition  and  emphasizes  change  in  a  dynamic 
environment.  That  of  Carretta  et  al.  implies  something  like  alertness  or  vigilance.  Houston  & 
Bruskiewicz’s  (2006)  definition  may  imply  a  three-dimensional  image.  In  any  case,  these 
definitions  differ  substantially  from  each  other  and  may  not  assess  the  same  attribute.  Because  of 
the  substantial  differences  in  the  definitions,  no  conclusions  can  be  drawn  from  these  studies 
about  the  importance  of  Situational  Awareness  for  success  as  a  military  pilot. 
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Knowledge 

Only  three  of  the  studies  were  concerned  with  identifying  the  types  of  knowledge  that  might 
discriminate  between  a  successful  and  an  unsuccessful  pilot  at  the  ab  initio  level.  Raters  in  Agee 
et  al.  (2009)  gave  aviation  knowledge  an  average  rating  of  4.75  out  of  5.0.  Youngling  et  al. 

(1977)  identified  aeronautical  knowledge  and  technical  vocabulary  as  critical.  Houston  & 
Bruskiewicz  (2006)  identified  four  categories  of  knowledge  that  potentially  could  be  used  to 
select  ab  initio  pilots:  principles  of  aviation,  meteorology,  aeronautical  terminology,  and  flight 
rules  and  regulations.  These  results  are  consistent  with  traditional  military  selection  processes, 
which  typically  assess  categories  of  aviation  knowledge. 

Traits 

Like  knowledge,  little  can  be  said  about  the  personality  traits  needed  for  success  as  a  pilot 
because  only  four  studies — Agee  et  al.  (2009),  Carretta  et  al.  (1993),  Houston  &  Bruskiewicz 
(2006),  and  Youngling  et  al.  (1977) — examined  them.  Any  discussion  of  important  traits  is 
limited  further  because  the  four  studies  used  different  approaches  to  generating  their  “other” 
characteristics  list.  Houston  and  Bruskiewicz  drew  many  of  their  traits  from  the  Five  Factor 
personality  theory;  Carretta  et  al.  may  have  made  limited  use  of  this  theory.  Agee  et  al.  generated 
an  initial  list  apparently  from  a  previous  Air  Force  study.  The  final  list,  however,  was  the  result 
of  additions,  deletions,  and  definitional  changes  made  by  SMEs.  The  result  was  that  the  lists  of 
“other”  and  “interpersonal”  attributes  share  no  attributes  with  those  of  the  other  three  studies. 
Youngling  et  al.  also  generated  their  list  from  SME  input,  as  well  as  from  a  literature  review  and 
an  examination  of  foreign  selection  systems.  Predictably,  few  attributes  are  shared  with  Carretta 
et  al.  and  Houston  and  Bruskiewicz. 

The  two  most  important  dimensions  of  the  Five  Factor  Theory  for  pilot  selection  appear  to  be 
Conscientiousness  and  Neuroticism  (Anesgart  &  Callister,  1999).  The  facets  of 
Conscientiousness  received  “very  important”  ratings  in  Houston  &  Bruskiewicz  (2006).  One  trait 
in  Carretta  et  al.  (1993),  Achievement  Motivation,  is  a  facet  of  Conscientiousness.  This  facet  was 
ranked  as  1  out  of  27  in  terms  of  importance  for  success  as  a  fighter  pilot.  Similarly,  the  facets 
related  to  Neuroticism  (Emotional  Stability)  received  ratings  of  “important”  in  Houston  and 
Bruskiewicz.  In  Carretta  et  al.,  Emotional  Stability  ranked  16  out  of  17.  Thus,  the  results  based 
on  the  Five  Factor  Theory  seem  consistent  across  these  two  studies. 

Interestingly,  Leadership,  Stress  Resistance,  and  Risk  Taking  were  considered  to  be  “critical”  by 
Youngling  et  al.’s  (1977)  raters.  Carretta  et  al.’s  (1993)  raters  ranked  Leadership  25  out  of  27 
and  Risk  Taking,  17  out  of  27.  The  four  leadership  attributes  and  Risk  Tolerance  in  Houston  & 
Bruskiewicz  (2006)  were  rated  as  “important,”  whereas  Stress  Tolerance  was  rated  as  “very 
important.” 

These  results  show  that  traits  related  to  Conscientiousness  are  considered  to  be  very  important 
for  success  as  a  pilot.  Traits  related  to  Emotional  Stability  are  considered  less  important.  Risk 
Tolerance,  Stress  Tolerance,  and  Leadership  show  mixed  results  but  are  seen  as  less  important 
than  traits  pertaining  to  Conscientiousness. 
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SUMMARY  AND  RECOMMENDATIONS 


Few  general  conclusions  can  be  drawn  from  these  studies  about  the  importance  of  specific 
attributes  because  of  the  definitional  problems  and  the  differing  uses  of  taxonomies  for 
identifying  KSAOs.  With  that  caveat,  the  two  most  important  abilities  are  Perceptual  Speed  and 
Spatial  Orientation.  Numerical/Quantitative  abilities,  Multi-Tasking  attributes,  Multi-Limb 
Coordination,  and  Selective  Attention  received  mixed  support.  Both  Mechanical  Aptitude  and 
Situational  Awareness  also  were  rated  highly.  However,  no  general  statement  about  the 
importance  of  these  attributes  can  be  made  because  of  differences  in  their  definitions. 

Three  recommendations  for  future  research  can  be  made.  The  first  concerns  the  taxonomy  used 
to  identify  the  required  KSAOs.  The  numerous  modifications  of  Fleishman’s  taxonomy  indicate 
that  the  taxonomy  is  not  sufficient  in  its  current  instantiation.  The  USAF  should  consider  the 
construction  of  an  expanded  taxonomy  that  would  include  personality  traits  and  other 
characteristics.  Currently,  the  0*NET  Content  Model  provides  an  expanded  taxonomy  of  non- 
cognitive  characteristics  relevant  to  civilian  occupations  (including  piloting)  that  may  provide  a 
useful  organizing  point  for  future  research  (National  Center  for  0*NET  Development,  n.d.).  For 
instance,  subject  matter  experts  in  one  or  more  of  the  studies  reviewed  in  this  report  identified 
the  following  five  0*NET  Work  Styles,  or  a  closely  related  construct,  as  relevant  to  military 
pilots:  Achievement/  Effort  (Carretta,  1993),  Adaptability/  Flexibility  (Agee  et  al.,  2009), 
Cooperation  (Agee  et  al.,  2009;  Carretta,  1993),  Integrity  (Agee  et  al.,  2009),  and  Self  Control 
(Agee  et  al.,  2009). 

Second,  more  research  on  Mechanical  Aptitude  is  needed.  The  relation  between  Mechanical 
Aptitude  and  the  spatial  abilities  needs  to  be  clarified.  The  nature  of  Mechanical  Aptitude  also 
needs  to  be  studied  to  determine  if  this  construct  is  composed  of  two  abilities  as  Carroll  (1993) 
suggests  or  only  one. 

Third,  research  on  Situational  Awareness  should  focus  on  addressing  problems  pertinent  to  pilot 
selection.  One  of  these  problems  concerns  the  nature  of  the  construct.  O’Brien  and  O’Hare 
(2007)  summarize  research  on  Situational  Awareness  and  suggest  that  it  has  a  hierarchical 
structure  with  basic  perceptual  abilities  at  the  lowest  level  and  higher  cognitive  abilities  at  the 
top.  The  nature  of  Situational  Awareness  (hierarchical  arrangement  versus  single  specific)  has 
implications  for  selecting  individuals  with  high  levels  of  the  construct.  If  Situational  Awareness 
reflects  multiple  abilities  arranged  in  a  hierarchical  manner,  the  underlying  abilities  need  to  be 
identified  and  their  relation  determined.  Another  problem  concerns  the  various  measures  of 
Situational  Awareness.  Questions  pertaining  to  exactly  what  is  being  measured  need  to  be 
addressed.  Additionally,  the  measures  need  to  be  examined  in  terms  of  selection.  That  is,  the 
reliability  and  validity  of  the  measures  must  be  established  and  the  presence  of  group  differences 
determined. 

On  the  whole,  the  results  of  this  review  provide  limited  insight  into  the  KSAOs  that  are  required 
for  successful  performance  as  a  pilot.  The  lack  of  progress  in  this  area  can  be  attributed  in  part  to 
problems  with  taxonomies  and  to  a  lack  of  understanding  of  the  nature  of  some  of  the  most 
promising  attributes.  A  concerted  effort  should  be  made  to  correct  these  deficiencies  in  a  timely 
manner. 
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